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[57] ABSTRACT 

A system is provided to automatically control input permis- 
sion when multiple users and/or software agents interact 
with a single, shared application in a window-based com- 
puting environment, and to indicate to all users which user 
or software agent currently is providing input or seeking 
input permission. The system operates by providing each 
user and agent with a specialized cursor icon that is used for 
displaying input-permission status and for requesting and 
relinquishing input-permission. Cursors have three modes 
with distinct appearances indicating whether the corre- 
sponding user or agent has. does not have, or desires input 
permission. Cursors are replicated on all displays, so that all 
users can at all times easily see who has or desires input 
permission. Users interact with the input-permission system 
by moving and clicking their mouse. Software agents 
achieve the same effects by sending control requests directly 
to a software interface provided as part of the invention. In 
one embodiment, each user and agent also has an assigned 
Home Window, which is used for identification and to 
compose and display text messages. 

12 Claims, 17 Drawing Sheets 
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MULTIPLE USER/AGENT WINDOW systems like SharedX do not replicate mouse pointers on all 

CONTROL users'displays. Instead SharedX provides separate tele- 
pointer icons which users have to explicitly click and drag 

This is a continuation of application Ser. No. 08/414.282 around, which is much more awkward than just pointing 

filed on Mar. 31. 1995 now abandoned. 5 with the mouse. 

The second major area of deficiency of current window 

FIELD OF THE INVENTION sharing systems is that, because they were only designed for 

~_. . ... • . - sharing between human users, they do not provide a well- 

This invention relates to the graphical user interface in * 5 *^ - * * n t*. * ♦ 

„. defined software interface to allow software agents to play 

window-based computing environments and more particu- in . , ^ „ t cJ 

. , _ . . . . 10 the role of remote users. For the present purposes, a software 

larly to controlling input to a window that is being shared . . i* -TL l 

. . acent is any program that interacts both with human users 

between two or more remote users or programs. * . ? p ,. .. , . 

r and some shared application program. In the future, software 

BACKGROUND OF THE INVENTION ncl P lincs ' for example, may provide an intelligent software 

agent instead of a human helper to assist users with simple 

It is often convenient for computer users at different 15 problems, 
locations to be able to view and interact simultaneously with 

a single window-based application program. For example. SUMMARY OF INVENTION 

when a user calls a software help line, rather than just Th e solution to the problems with the prior art described 

describing the problem in words, the user can actually above is. first, to provide a software interface whereby 

demonstrate the problem to a remote helper using a shared 20 software agents can participate in window sharing on the 

window. During the process, the remote helper can also samc 5^ & human userSt second, to provide an automatic 

provide input to the same window to help in the diagnosis. input-permission system that allows at most one user or 

The current state of the art in this kind of remote window software agent to have input permission at a time, and third, 

sharing is typified by Hewlett packard's SharedX. Hp prod- to provide clear indication on all displays of who is currently 

uct Number B2305 L. or the Xy system described in "Xmc 23 providing input 

and Xy— Scalable Window Sharing and Mobility." C. The automatic input-permission system allows at most 
Bormann. and G. Hoffman. 8th Annual X Technical o DC uscr or software agent to have input permission at a 
Conference. Boston. Mass.. January. 1994. time, but makes it easy for users and agents to obtain and 
These systems are extensions to the X Window System relinquish input permission without interrupting the flow of 
that allow real-time sharing ofX-protocol based applications 30 collaboration. This is achieved by providing each user or 
between two or more remote users or displays. In such software agent with a single cursor icon on die shared 
systems, the shared application window is automatically window that is used for pointing, for inputting, for display- 
replicated on all users'displays, so that all users can simul- ing input-permission status, and for requesting and relin- 
taneously see changes in the state of the application. quishing inputpermission. Each cursor is replicated on all 
However, only users who have input permission are allowed 33 displays. 

to provide input to the application, e.g.. by clicking with a in order to distinguish identity, each cursor has a different 

mouse or using a keyboard. Any number of users can have basic appearance, such as a different color or shape. For 

input permission at the same time, with input permissions example, in one embodiment of the invention, each cursor is 

displayed and changed via toggle buttons next to in a different color pointing hand icon, 

users'names on a separate control window. 40 m ^dcr to support software agents as well as human 

Two major areas of deficiency in SharedX and similar users, in one embodiment the automatic input-permission 
systems are described below. First, the graphical user inter- system is divided into two modules: a controller and a 
face in current window sharing systems is confusing and translator. The controller is a program that receives control 
awkward. The main problem, acknowledged in Chapter 4 of 4J requests, e.g.. requests to grant or relinquish input 
the SharedX 2.0 User's Guide, is that when sharing a permission, from other programs. Software agents send 
window with several remote users who have input control requests directly to die controller's software inter- 
permission, it is confusing to tell who is inputting at any face. The job of the translator component is to translate a 
given time. user's cursor actions into control requests according to the 

For example, suppose three users are sharing the same x rules described below, 

application window, such as a spreadsheet or drawing pro- in one embodiment, the automatic input-permission sys- 

granx If two of the users both have input permission and one tern operates as follows. Each cursor has three modes, also 

of them pushes a button on the shared window, the third user distinguished by appearance. When a cursor is in the Active 

will see the button move, but will not be able to tell from his mode, it means that the corresponding user or software agent 

display who pressed iL Furthermore, if two users with input 55 has input permission. In the passive mode, the corrcspood- 

permisskm both start pressing buttons at the same time, they ing user or agent does not have input permission. In the 

are likely to interfere with each other's work. attention-getting Signalling mode, the corresponding user or 

One solution to this problem would be to only let one user agent desires input permission, ix.. would like whoever 

have input permission at a time. However, that is not the way currently has input permission to relinquish it. 

that current systems are usually used, because changing $o For example, in one embodiment of the invention in 

input permissions requires going to a separate control which each cursor is a different color pointing hand icon, 

window, which is inconvenient and interrupts the flow of the when the cursor is Active, the hand is a solid color; when it 

collaboration between users. is passive, the hand is only an outline; and when it is 

Moreover, when two people are viewing a display in the Signalling, the outline hand blinks or oscillates in order to 

traditional way, i.e.. standing or sitting side by side, they 65 get the other users'attention. 

often point to items on the display with their fingers, or use Thus at all times it is easy for all users to see who has or 

the mouse to point out items. Unfortunately, window sharing desires input permission simply by looking at the appear- 
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ancc of the cursors on the shared window. Software agents 
can find out who has or desires input permission by sending 
the appropriate control requests to the controller module. 

Human users move their cursors and provide input simply 
by moving and clicking their mouse, i.e.. the cursor replaces 
the usual mouse pointer on the shared window. Because 
cursors are replicated, they can also be used for pointing out 
items to remote users, making separate telepointers unnec- 
essary. 

A software agent does not have a display or mouse. As a 
result, software agents move their cursors and provide input 
by sending the appropriate control requests directly to the 
controller module. 

All cursors start out in passive mode located off the shared 
window. When a user or software agent moves his cursor 
onto the shared window, it remains in the passive mode until 
the user or agent attempts to provide input, e.g.. until the 
user presses a mouse button or key. At this point, the system 
automatically grants input permission to that user or soft- 
ware agent, i.e.. makes the cursor Active, if and only if no 
one else is Active. If the user or agent's cursor is not made 
Active, then it is put into Signalling mode. 

In one embodiment, a user or software agent relinquishes 
input permission simply by moving his cursor off the shared 
window. Moreover, a user can force the Active user or 
software agent to relinquish input permission by clicking on 
that user or agent's cursor. A software agent can force the 
Active user or agent to relinquish input permission by 
sending the appropriate control request to the controller 
module. 

In one embodiment, each user and software agent is 
provided with a special window, called a Home Window, 
which is a place for the user or agent to put his cursor when 
he does not want to point or provide input Like cursors. 
Home Windows are replicated on all displays. Home Win- 
dows assist in identification because each Home Window is 
labelled with the name and possibly the face of the corre- 
sponding user or software agent, and is coded with, for 
example, the same color as the corresponding cursor. 

Another function of Home Windows is to support com- 
munication between users and software agents. Typically, in 
addition to sharing windows, human users are also in voice 
communication, e.g.. by telephone. Software agents, 
however, typically cannot speak or hear, and therefore need 
some other mechanism for communicating with users. 
Human users and software agents can therefore display text 
messages in their Home Windows, which since they are 
replicated on all displays, will be seen by all users. Users can 
compose messages either by typing the text or by selecting 
from a menu of predefined messages. Agents can read 
users'displayed messages by sending the appropriate control 
requests to the controller module. As an additional 
convenience, users can force another user or agent to relin- 
quish input permission by clicking on that user or agent's 
Home Window. 

In summary, a system is provided to automatically control 
input permission when multiple users and/or software agents 
interact with a single, shared application in a window-based 
computing environment, and to indicate to all users which 
user or software agent currently is providing input or seek- 
ing input permission. The system operates by providing each 
user and agent with a specialized cursor icon that is used for 
displaying input-permission status and for requesting and 
relinquishing input-permission. Cursors have three modes 
with distinct appearances indicating whether the corre- 
sponding user or agent has. does not have, or desires input 
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permission. Cursors are replicated on all displays, so that all 
users can at all times easily see who has or desires input 
permission. Users interact with the input-permission system 
by moving and clicking their mouse. Software agents 
5 achieve the same effects by sending control requests directly 
to a software interface provided as part of the invention. In 
one embodiment, each user and agent also has an assigned 
Home Window, which is used for identification and to 
compose and display text messages. 

10 

BRIEF DESCRIPTION OF DRAWINGS 

These and other features of the Subject Invention will be 
better understood in conjunction with the Detailed Descrip- 
tion taken in conjunction with the Drawings of which: 

15 FIG. 1 is a diagrammatic illustration of a typical prior art 
system, showing a shared, replicated application window 
and an input-permission control window, where one user 
cannot tell from his display which of the other remote users 
is currently providing input to the shared window; 

20 FIG. 2 is a dUagrarnmatic illustration showing the use of 
the Subject Invention in the same situation as FIG. 1. with 
the addition of an automatic input-permission system and 
replicated cursors, so that a user can easily see which remote 

y user has pressed a particular button, and the addition of a 
software agent participating in the collaboration on the same 
basis as the human users; 

FIGS. 3A and 3B are diagrammatic illustrations showing 
how a user is automatically granted input permission simply 

30 by clicking his mouse, given that no other user or agent has 
input permission; 

FIG. 4 is a diagrammatic illustration of the windows seen 
on all users' displays when Home Windows are used; 
FIG. S is a block diagram of the automatic input- 

35 permission system of FIG. 2. showing its decomposition in 
one embodiment into a translator module and a controller 
module with associated controller state; 

FIG. 6 is a flowchart showing the top-level logic of the 
controller module, which handles ten types of control 

40 requests; 

FIG. 7 is the detailed flowchart for the processing of an 
Activate request by the controller module; 

FIG. 8A is the detailed flowchart for the processing of an 
Deactivate request by the controller module; 

FIG. 8B is the detailed flowchart for the processing of a 
Signal request by the controller module; 

FIG. 9 is the detailed flowchart for processing of an 
Interrupt request by the controller module; 
50 FIG. 10 is the detailed flowchart for processing of a Move 
request by the controller module; 

FIG. 11 is the detailed flowchart for processing of an 
Execute request by the controller module; 

FIG. 12A is the detailed flowchart for processing of a 
55 Status request by the controller module; 

FIG. 12B is the detailed flowchart for processing of a 
Show request by the controller module; 

FIG. 13A is the detailed flowchart for processing of a 
M Clear request by the controller module; 

FIG. 13B is the detailed flowchart for processing of a 
Read request by the controller module; 

FIG. 14 is a flowchart showing the top-level logic of the 
translator module, which handles four types of window 
£5 events resulting from a user's cursor actions; 

FIG. ISA is the detailed flowchart for processing of an 
EnterWindow event by the translator module; 
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FIG. 1SB is the detailed flowchart for processing of a Referring now to FIG. 3A. when no user or software agent 

LeaveWindow event by the translator module; has input permission, all of the cursor icons on window 14 

FIG. 16 is the detailed flowchart for processing of an appear in outline. Referring now to FIG. 3B. a user 22 has 

Input event by the translator module; and clicked on a button 24 of a mouse 26. the result being that 

FIG. 17 is the detailed flowchart for processing of a 5 corresponding cursor icon 52 has changed to being filled 

pointerMotion event by the translator module. Jn ' the user 22 has rcceived m P ut P™««»- 

Referring now to FIG- 4. in one embodiment each user 

DETAILED DESCRIPTION and software agent is assigned a Home Window 64. 66, 68. 

and 70. where the corresponding cursor icon appears when 

Referring now to FIG. 1. in the prior art a network 10 10 it ^ DOt 0D the sha red window. Home Windows are repli- 

connects computer nodes 12 such that windows 14. 14'. and catcd on ^ ^^ ySm Moreover, a title bar 65, 67. 69. and 71. 

14" are duplicate window interfaces to a single shared on each Home window can be used to display the name of 

application program. As is common, for example in the ^ corresponding user or software agent. 

P™*?-. an prated control! *indow HcfC can ^ sccn mat a user ^ ^ moved M , cursor 

40. 40 and 40 is provided to indicate and modify which 15 ioon fR)m ^ Home ^ QQtQ ^ shared window M 

users have input permission. However, when more than one and d ^ ssion . ^ me usex wiln 

user has input permission, there is no indication of which window 64 is indicating his desire to obtain input 

user is actually providing inpu atany given moment This b $i ^ with osdllati C(M H ome 

makes it is more difficult to collaborate, since users cannot LT r , . 6 _ , u j. , * . . 

it ' h rf'd h Windows also support (he display of text messages to 

ascertain wno w . 20 jupp^ collaboration between users and software agents. 

For example, a user 16 looking at window 14 at node 18 0nc such tcxt me ssage is illustrated at 71 

is in a quandry as to which of the other users on a network Rcfcni n(jw tQ nG $ fa one cmbodiment ^ ^ 

10 has caused the depression of a button 20 as indicated. As ^ iBputi „ misskui system 50 includes a translator mod- 

.Uus*ated.ase^ ^ M tQ a C0Qtroller 82 wnjcn m turQ 

of the button 20 by clidang • button 24 of his mouse 26. 25 ^ ^ $ a cootroUcr state ^ ^ As ^ 

This user 22 can see which button of the shared window he ^ sccn a netW£>rk 10 window event& from ^ 

■s pressing ; by vutue of the location of a pointer 28 on to ^ ^ which 

window 14 whose portion is controlled by mouse 2& ^ ^ ovcr ^ w f<Jf wccutiofl b me 

However, this pointer 28 does not show up on window 14 * ontrolIer moduJe jg. 

which user 16 is looking at 50 

, , , _ . . The controller module 82 receives control requests from 

At nearly the same time, another user 30 is attempting either At translator module 80 or directly from software 

through his associated mouse 32 and button 34 to actuate a w The controller module 82 grants and retracts input 

different button 36 as indicated by pointer 38 on window ^ welJ as producmg window events transmitted 

14^ '.In such a situation.it quickly becomes hard for all users a networft 10 t0 u ^ ^ windows on aU the 

to figure out what is going on. 35 users'displays. 

In an effort to make it easier for users to share windows B of mustrat ion, suppose a user 22 clicks a button 

without confusion, referring now to FIG. 2, network 10 is 24 on his mouse 26 at a moment when his cursor is located 

provided with an automatic input-permission system 50 on {op of another uscr ^ The resulting 

which both selects which user has input permission and wiQdow event is tansndtted me nctwork 10 to the 

displays that information to all users through icons on the translator module 80. where it is translated into an Interrupt 

shared window. For instance, assuming that a user 22 has ^ ie a uest t0 force me other uscr to Kliliqtlish 

been granted input permission by automatic input- . t The Interrupt request is then sent to the 

permission system 50. a unique cursor icon, in this case a module 82. which checks whether the other user 

solid upward and leftward pointing hand 52, is displayed on 45 or t - m fact docs currcnUy havc mpu( con£roL ^ tf u 

all windows at the location at which input will occur. Thus docs . appropriately updates the controller state 84 and 

at any given time, it is clear to all users which user or transmits though ^ network 10 the appropriate window 

software agent has input permission. events to change the other user's cursor appearance 00 all 

In the depicted embodiment, the correspondence between displays, 

a cursor icon and a user or software agent is indicated by the x The rules by which the controller module grants and 

orientation of the hand. In another embodiment the color of re tracts input permission are carried by the data structure 84 

the cursor icon is used to uniquely identify the correspond- interacting with the program stored in the controller module 

ing user or software agent. 82- As illustrated, for each user or software agent, there are 

Id the depicted embodiment, cursors 52. 54. and 56 two pieces of information stored within the data structure 84. 

belong to the human users shown in the lower right, upper 55 namely location and mode. Location refers to the location of 

right, and lower left of the figure, respectively. Cursor 58 that user or software agent's cursor icon, which is either 

belongs to a software agent 60 depicted in the upper left of Empty, meaning that the cursor is not on the shared window, 

the figure. Using a network 10 and the automatic input- a particular x-y location on the shared window or, in an 

permission system 50, a software agent 60 competes for the embodiment with Home Windows, on that user or software 

opportunity to provide input to the shared application in «, agent's Home Window. Mode refers to whether the user or 

much the same way as the human users. software agent has input permission. In the Active mode, the 

In the depicted embodiment, the cursor of the user or user or software agent has input permission. In the passive 

agent, if any. which currently has input permission is ren- mode, the user or software agent does not have input 

dered as a solid hand, while all the other cursors are rendered permission. In the Passive mode, the user or software agent 

in outline only. Notice that cursor 58 is oscillating, which is 65 does not have input permission. In the attention-getting 

an attention-getting signal to the other users that the corre- Signalling mode, the user or agent does not have input 

s ponding user or software agent desires input permission. permission and desires input permission. 
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It will be appreciated that a software agent 60 is connected 
through the network 10 to the automatic input-permission 
system 50 via line 88 on which control requests are trans- 
mitted directly to the controller module 82, in contradistinc- 
tion to the human users who send window events to the 
translator module 90. Software agents can determine the 
result of their requests to the controller module 82 by 
examining the return values that are transmitted via line 90 
from the controller module to the software agent. 

FIG. 6 is a high-level flowchart for the processing in one 
embodiment of the controller module 82. Upon receipt of a 
request, the controller determines which of the ten valid 
requests types has been received and dispatches to the 
appropriate processing routine described further in the indi- 
cated figures. The ten decision blocks in this figure check in 
turn for the receipt of an Activate request a Deactivate 
request, a Signal request, an Interrupt request for a given 
user or agent a Move request to given location, an Execute 
request of a given input, a Status request, a Show request for 
given message, a Clear request, or a Read request for given 
user or agent 

FIG. 7 is a detailed flowchart for the processing of an 
Activate request. A software agent would send this type of 
request when it wants to provide input to the shared appli- 
cation window. This request type is automatically generated 
for users by the translator module whenever a user tries to 
provide input. If the user or software agent mat issued the 
Activate request is located off the shared window or another 
user or agent is currently Active, then the requestor's cursor 
mode is set to Signalling and the request fails; otherwise, the 
requestor's cursor mode is set to Active and the request 
succeeds. 

FIGS. 8A is a detailed flowchart for the processing of 
Deactivate requests. A software agent would send this type 
of request when it was done providing input This type of 
request is automatically generated for users by the translator 
module whenever a user leaves the shared window. The 
effect of a Deactivate request is simply to set the cursor 
mode of the requesting user or software agent to passive. 

FIGS. SB is a detailed flowchart for the processing of 
Signal requests. A software agent would send this type of 
request when it wanted to get the attention of the other users 
and/or agents, for example, so it could begin providing 
input. In embodiments with Home Windows, this type of 
request is automatically generated for users by the translator 
module whenever a user clicks the mouse on his own Home 
Window. The effect of a Signal request is to set the cursor 
mode of the requesting user or software agent to Signalling, 
which causes the corresponding cursor icon to oscillate or 
blink on all displays. A user or software agent's cursor mode 
is also set to Signalling whenever an Activate request fails, 
as shown in FIG. 7. 

FIG. 9 is a detailed flowchart for the processing of an 
Interrupt request This type of request would most typically 
be used by human users to take input permission away from 
software agents. In fact in some embodiments, the auto- 
matic input-permission system may restrict who can inter- 
rupt whom. The translator module generates an Interrupt 
request, as shown in 16, whenever a human user clicks on 
top of another user or software agent's cursor. If that user or 
software agent's cursor is Active, the cursor's mode is set to 
passive and the request succeeds; otherwise the request fails. 
In embodiments with Home Windows, the translator module 
also generates an Interrupt request whenever a human user 
clicks on a Home Window other than his own. 

FIG. 10 is a detailed flowchart for the processing of a 
Move request. Move requests are automatically generated 
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for human users by the translator module in response to then- 
mouse motions, so that their cursor icons appear to behave 
the same as their usual mouse pointers. Since software 
agents don't have mice, they move their cursors by sending 

5 Move requests directly to the controller module. If the 
requesting user or software agent' s cursor is currently on the 
shared window and the new location is also on the shared 
window, then the requestor's cursor location is simply 
updated to the new location. If the requestor's cursor is 

10 moving from off the shared window to on or on the shared 
window to off. then the requestor's cursor mode is first set 
to passive. In either case, the Move request succeeds. 

FIG. 11 is a detailed flowchart for the processing of an 
Execute request This request type is used by software 

15 agents to provide input, such as mouse and keyboard press 
and and release events, to the shared application window. 
From the standpoint of the shared application, input from a 
software agent looks just like input from any of the users. 
Execute requests are automatically generated for users by 

20 the translator module, as shown in FIG. 16. If the requesting 
user or software agent is Active, then the given input event 
is sent to the shared window and the request succeeds; 
otherwise the processing for an Activate request, shown in 
FIG. 7. is performed first. If the Activate processing fails. 

25 then the Execute request fails. 

FIG. 12A is a detailed flowchart for the processing of a 
Status request which is used by software agents to find out 
about the location and mode of other users' and software 
agents 'cursors because, unlike human users, software agents 

30 cannot simply look at their displays. A Status request returns 
the complete state of the controller in an appropriate data 
structure that can be interpreted by the requesting software 
agent 

FIG- 12B is a detailed flowchart for the processing of a 
35 Show request which is used in embodiments with Home 
Windows only. This request is most likely to be used by 
software agents to communicate with human users. The 
Show request generates appropriate window events to dis- 
play the given text message in the requesting user or 
40 software agent's Home Window. Since Home Windows are 
replicated, this information is displayed to all users. In one 
embodiment, human users can generate a Show request by 
interacting with a menu or text editor in their Home Win- 
ds dow " 

FIG- 13A is a detailed flowchart for the processing of a 
Clear request which is used in embodiments with Home 
Windows only. The Gear request generates appropriate 
window events to erase any messages currently appearing in 

3Q the requesting user or software agent's Home Window. 
FIG. 13A is a detailed flowchart for the processing of a 
Read request, which is used by software agents in embodi- 
ments with Home Windows to read messages from other 
users or software agents. If the given user or software agent 

55 currently has a text message displayed, then that message is 
returned to the requesting user or software agent; otherwise 
the request fails. 

FIG. 14 is a high-level flowchart for the processing in one 
embodiment of the translator module 80. Upon receipt of a 

60 window event from a user's display, the translator deter- 
mines which of four relevant window event types has been 
received and dispatches to the appropriate processing rou- 
tine described further in the indicated figures. These basic 
purpose of these processing routines is to send the appro- 

65 priate requests to the controller module. The four decision 
blocks in this figure check in turn for the receipt of an 
EnterWindow event at a given x-y location, a Leave Window 
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event, an Input event, or a pointerMotion event at a given FIG. 16 is a detailed flowchart far the processing of an 

x-y location. These are four generic event types in typical input event Some Input events, such as clicking on another 

window systems- EnterWindow and LeaveWindow are the uscr ' s curs0 r to generate an Interrupt request, are used to 

events generated when the pointer enters and leaves a interact with the automatic input-permission module. Other 

window, respectively. pointerMotion is the event generated 5 Input cycnts such as kcyboard even ts. are in effect passed 

whenever the pointer moves. Input is the event generated through to fo c application program. In embodiments with 

when the user presses or releases a mouse button or key. Homc windows ^ tne usef . s is 0D ms own or another 

FIG. 15A is a detailed flowchart for the processing of an user „ software agent's Home Window, then either a Signal 

EnterWindow event. While a user is moving bis pointer or an Iruerrupt request is generated, respectively; otherwise, 

around in windows other than the shared application " tf ^ uscf$ h CQ of mo&iet thcn an 

window. h,s cursor location is either Empty, or in ernbodi- ^ t fa ^ wftft ^ othcr cursof . s owncr 

ments with Home Windows, at Home. Whenever the user s M ^ H tf ^ of ^ aboye ^ ^ ^ 

pointer enters the shared window, however, a Move request * ^ . & ^ ^ £ as 

is generated, which updates his cursor s location in the , . n . ii . 6 ii . 5, 

controller state to the^ven x-y location, and causes his » argurnenC Assunung that that the requesting , user or software 

corresponding cursor icon to appear on the shared window. a « eQt has or can "* ^ tcd m P ut P™ 551 ™' * e ^ 

Notice that simply entering the shared window does not r «l ucst wul * Cfl P* ss ^ "P 1 " on to thc app^ation 

translate into a request for input permission. Input permis- program. 

sion is not requested until the user actually clicks or presses nG yj is a ^tailed flowchart for the processing of a 

a key on the shared window. 2° pointerMotion event, in which a Move request is generated 

FIG. 15B is a detailed flowchart for the processing of a f or the given x-y location. While a uscr's cursor is located 

LeaveWindow event, in which a Deactivate request is gen- on lne shared window, his cursor icon tracks his mouse 

erated. In embodiments with Home Windows, a Move mo Uons just like the usual mouse pointer, 
request is also generated to move the user's cursor to his 

Home Window. Thus regardless of where else on his display 25 Below is an emfcxiiroent of the Subject Invention written 

his pointer is located, once a user leaves the shared window. in Hp Common Lisp 4.0.3 from Harlequin, Inc. using the 

from the standpoint of other remote users, his cursor appears Garnet 2. 1 graphics package from Carnegie Mellon Univcr- 

on his Home Window. sity. 
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! ;;; Code 1 or Window Application Sharing 

l| ; ; »*»*»***♦»♦♦**»*•*♦ + ♦•***»*♦♦♦♦»•♦♦***♦**»•***«**•**•••********* 

f' (defparaaeter *window-h« ight* 300 "Height of application window") 

j : (del parameter *window-uidth* «00 "Width of application window") 

!. (defvar *x-window* nil "X window for application 1 *) 

j' (defvar *x-root« nil "Root of *jc-window*") 

jj (defvar *panel* nil "Application panel on *r-wlndow»' 1 ) 

I- (defvar +agents* nil "List of agents") 

(defvar *self» nil "Bound to an agent in3lda each agent process") 



(defvar *display-host* 

(opal: ;get-diBplaj-name (environment- variable "DISPLAY")) 
"Currant display host name") 



|! (dnf parameter •agent-priority* 100 "Priority of software agent processes") 

]': (defparameter ♦local-priority* SO "Priority of local interface processes") 

i ! 

j) (def struct C agent 

I; ( :print-function 

i[ (lambda (agent stream depth) 

ii (declare (ignore depth)) 

j! (format strea» "KAgenT ~A>" (*g#iit-iiM« agent))))) 

'■ (name "") ;unique string that names agent 

|; (display nil) ;display used by agent (if human) 

i! (window nil) ; appl ication window (interactor-window) 

|; (window-id nil) ;X server id of window 

>'• (others nil) :* windows for other agents' remote hones 

\\ (panel nil) ;application gadget in window 

'• (local nil) ;local home window (interaetor-window) 

\'. (local-id nil) :* server id of local hone window 

l! (remotes nil) ;remote hone windows (interaetor-window) 

!' (x 0) ;* location of hot spot of cursor 

j (y 0) ;y location of hot spot of cursor 

I. (cursor nil) ;agent'o on-window cursor (cursor) 

; l (home-cursor nil) ;agent's home cursor (home-cursor) 

jj (urgent nil) ;t iff agent is in urgent state 

jj (oscillate-process nil) ;procoss controlling oscillating 

'l' (proceas nil) ; agent process 

!j ) 

jl (defvar *host-display* opal : :*aef ault-X-display*) 

j! (defvar *ohow* t "To inhibit show requests") 

jj 

|i (defun init (koptional dick-host (teit t)) 

\[ (kill) 

!■ (aetq »ahow* text) 

I! (if dick-host 

j) (create-agent "Dick" dick-host ' :top-left dick-face nil 

l! - 17 ■ 
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user: : ♦rightdowi-f ill* user: : *rightdovn+ 
user : : *rightdovr.-oscil late* 
opal; blua text) 
(creatw-agant "Saurk" nil : top-left smurk-face 'snurX-loop 
user: : *rightdown-f ill* user: :*rightdown* 
user: : *rtghtdown-oscillate* 
opal: blue te«t)) 

(create- agent "Chuck" (check* «display-host*) : bottom chuck-face nil 

user: :»rightup-f ill* user : : *rightup* user : : *rightup-oscillate* 

medium-green tart) 
(create -agent "Glurk" nil :top-right glurk-fac* 'glurk-loop 

user: : *lef tdown-fill« user: :»leftdovn* user: :*lef tdovn-osr.niat« 

opal : red text) 
<s«tq *st»urk-likes* (find-button "Buttonl") ) 
(setq *snurk -hates* (find -button "Button7")) 
(setq *glurk-hat*»s* (find-button "Butt.on6")) 
(satq *glurk-like3* (find-button "Buttons")) 
(init-agenta) 
) 

(defun init-agenta () 
(update) 

start processes at end after all windows napped 
(dolist (agent *agents*) 

(let ((loop (agent-proceas agent)) 

(display (agent-display agent)) 

(agent agent)} ;fii compiler bug 

(whan (or loop display) 

(setf (agent-process agent) 
(make-process 
:naae ( agent -naae agent) 

:priority (if display *local-priority* ♦agent-priority*) 

: function 

i'UajEbda () 

(when display (flush-display display)) 
(handler-bind 
((error t' invoke -debugger ) ) 
(let ((*self* agent) 

(inter : *enter-notif y-hook* ' enter-notify) 
(inter : *leeve-notif y-hook* * leave-notify) 
( inter : *mot ion-notif y-hook* ' cursor-notion) ) 
(loop 
(if display 

(inter: : def ault-e» ent-handler display) 
(funeall loop))))))))))) 

t) 

(def tin kill 0 

(satq 'panel* nil) ;tum off agent processes 
(doliat (agent *agents+) 

(let ((oscillate (agant-oscillate-procesB agent)) 
(process (agent -process agent))) 
(when (processp oscillate) (kill -process oscillate)) 

. |S- 
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i (when (procossp process) (kill-process process)) 

(if-let (window (agent -window agent)) 
(progn (opal : destroy window) 

(opal: destroy (agent-local agent)))) 
(mapc opal: destroy ( agent -reaotea agent)))) 
(clean) 

(dolist (agent *agants*) 
• ! (close-if-l ive (agent-display agent))) 

(setq *aganto» nil chuck nil dick nil glurk nil . 
,■ *x-t»indow* nil «x-root» nil 

♦active* nil •passive* nil *lock» nil 

+smurk-likes» nil *saurk-hates* nil 'glurk-likes* nil «glurk-hat»a* nil 
•user-send* nil *trace-agents* nil)) 

|[ 

: ' (defun ereate-agent (name display-nan" location face loop 

I active passive urgent 

; toptional (toraground-color dark-groan) (text t)) 

j (let» ((master (creata-instance nil cursor 

V (: active- image active) 

!' ( : passive- image passive) 

[ (:for«ground-color foreground-color) 

(:hot-x (or (xlib : inage-x-hot active) 0)) 
!i (:hot-y (or (xlibiiaage-y-hot active) 0)) 

1 ! <:visible nil))) 

!j (display (when display-name (open-opal -display display -name ) )) 

|' (agent (nake- agent :nua nana : cursor natter : display display))) 

j| (set (intern (string-opcase name)) agent) 
j; (when display 

|l (create-application agent display active passive foreground-color)) 

-i (local-home agent location face passive urgent foreground-color te*t> 
|i (s-v&lue (agent-home-euroor agent) :vlsible 

II (o-foraula (not (gv (agwit-cursor agent) rvisible)) t)} 
!■ ;; create slaved hoae windows and cursors 

ij (if -let (window (agent-Hindoo agent)) 

(let ((agg (g-value window aggregate))) 
j] (doliet (other *agents«) 

!! (progn (push (ramote-hooie other window) (agent-others agent)) 

;j (opal :add-coaponont agg (create-instanca nil 

I; (agent -cursor other))))))) 

| ! (dolist (other ♦agents*) 

j; (if-let (window (agent-window other)) 

ji (progn (push (remote-home agent window) (agent-others other)) 

j! (opal :add-conponent (g-valuo window : aggregate) 

) : (create- instance nil raster))))) 

!: (push agent *agents*) ;aust be done here 

|! (setf (agent-process agent) loop))) 

[j (defmacro eith-opal-display (display abody body) 

1 '(let* ((opal: :*default-x-display« .display) 

|i (opal: :*default-x-acreen* 

■i (first (xlib:display-roots opal: : *def ault-x-display*) )) 

!• (opal: :*delault-x-root* (xlib : screen-root opal : : *def ault-x-ecreen*) ) 

i! (opal: : •default-x-colonsap* 

r 

!' • ]fl 
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(xlib:screen-default-colorii8p opal: :*d«fajlt-x-scre<sn*))) | 

!i . .body)) j 

' j I 

jj ■ •■ Note assuming that there is at least one human user with display J 

I 

'•' (defun creata-application (agent display active paasioe foreground-color) . 

,| ; ; returns display ! 

!j (1*t.» ((host (xlibrdisplay-host display)) j 
: (window (create-inatance nil inter: interaetor-windou 

! | (:top (if (sow* j 

(lambda (agent) j 

j! (if-l«t (d (agent-display agint)) j 

\\ (equal (xlib:display-host d) host))) j 

!! * agents*) j 

jj 600 100)) I 



I i 



left (half C- 1280 *windov-width*))) 

title "Application") j 
agent agent) 
doubls-buf f«red-p t) 
height »window-height*) 

width *window-width*))) j 
(top (create-instance nil top -aggregate) ) . J 
' i (panel (create-instance nil demo-panel I 

j (: agent agent)))) j 

j: (s-value panel : value "Button?") j 
, (opal : add-conponent top panel) 
] (s-value window aggregate top) 

(setf (agant-windov agent) window) 
!| (setf (agent-panel agent) panel) 
j: (with-opal-display display 
!< (opal: update window) 

!] (s-v&lue window : background-color 

j! ( display- color-get window opal :notif-gray) ) 

Ij (opal: update window)) 

ij (create-instance nil inter :buttcn-int«ractor \ 
1 : (: window window) i 

jj (: agent agent) 

!; (:start-where ' (:in , window)) j 

ji (continuous nil) j 

jj ( :final-f unction 'click-final-! unction)) j 

j; (let ((drawable (draw able window)) j 
j; (root (display-root display)) 

j; (screen (display-screen display)) 

ji (color (diapl ay-color-get window foreground- color) )) 

j| (setf (xlib:window-event-»ask drawable) inter ;: *report-motion-em*) 

I: (setf (getf (ilib:window-plist drawable) 'active-r-cursor) | 

j; (craata-x-cursor active active root screen color)) • 

i| (setf (get? (xlib:window-plist drawable) 'p&ssive-x-cursor) j 

ji (create-i-cursor passive passive root screen color)) i 

j- (unless 'panel* j 

ij (oetq ♦panol* panel 

jj »x-windov* drawable 

jj »i-root* (display-root display))) 
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(setf (ag«nt-windou-id agent) (xl ib:windo»-ld drawable))} 
display) ) 

Software Interface: Controller Requests 

ji .;; .............. 

'! (defvar tactive* nil "Agent that la active, if any") 

|i (defvar *passive* nil "List of agents that are on.-vindos . but passive, if any") 

j! 

i] ;;; Note for simplicity in this application va are sequentializing 

I ;;; requests by locking calling processes, rather than queuing. 

j| (defvar *lock* nil "Lock for controller requests") 

ji ;;; Agent, issuing request (*self*) is implicit argument to each request. 

J- (dofvar *trace-agents* nil "List of agents whoso requests are to be traced") 

M 

j! (defun trace-agents (trest agents) 
|) (dolist (agent agents) 

I I (puetuies agent •trace-agents*)) 
ji *trace-agents*) 

ji 

.1 (defun untrace-agents O 

jj (setq *traca-agents* nil)) 

(1 (dofaacro defrequest (name args fcbody body) 

|i (let ((trace (gensym)) 

j! (value (gensym))) 

| : '(defun .name , args 

l! (let ((.trace (member *oelf> ♦ trace-agents*))) 

j! (Bhen .trace (format t ""V :(~A~): *A'«{ *A">" (agent-name "self*) '.name 

jj • .args)) 

j| (let (C, value (progn . .body))) 

j! (when .trace (fornat t " «> ~A" .value)) 

|j .value))))) 
[ 

jj (defrequest ACTIVATE () 

j! Request control of shared slndon. 

j I ;; Succeeds iff agent is on window and no other agent is active, 

i; ;; After successful completion, agent is active, 

jj (unless (at-home? *self«) 

(slth-process-lock (*lock*) 

{' (cond (*active» (eq 'active* *self*)) 

Ji (t (setq •active* *self*) 

j, (s-ualue (agent-cursor *self«) :active t) 

jj (if-let (window (agent-window *aelf*)) 

jj (let ((drawable (dravable Hindoo))) 

j' (set-cursor drawable 

!| (g«tf (xlib:vindow-pli*t dravable) 

ji *active-x-cursor)))) 

|; (setq *paasivo* (delete -s«lf* 'passive*)) 

I: 
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(upd at a- windows) 
t))))) 

(defraquest DEACTIVATE () 

Relinquish control of shared uindou. 
Always succeeds. 

If agent is on window, then agent is passive at end of request, 
otherwise request has no effect, 
(uith-prococs-locli (*lock») 
(when (eq *activ«* *self*) 
(deactivate-internal ♦s*if+) 
Cpushneu *self* «passiTe*) 
(update-windows) ) ) 

t) 

(dafun deactivate-internal (agent) 
(setq "active* nil) 

(s-value (agent-cursor agent) :actire nil) 
t) 

(defrequest HOVE (z y) 

Hove agent's cursor to given x,y location on shared window. 
Always succeeds. 

If agent is at hone when request issued, then agent is passive at end 

of request. 

[If agent is on window, cursor aode is unchanged.] 
{uhen (at-hoae? *self*) 

(uith-process-loci (*lock») 
i: (when * active* 

j; (s-value (agent-cursor »self») :aetive nil) 

i 1 (pustinew tself* ^passive*))) 

jj (oscillate *aell* nil) ;aaa hone 

start cursor under (original location of) home 
|; (let ((location (g-value (agent-hose* cursor *self*) : location))) 
j' (warp-cursor *aelf* 

[i (ecase location 

l! ((:top-left :&ottoi»-left) O) 

|| ((:top-right :tx>ttos-right) *window- width*) 

i, (:bottoa (half *window-width*3 )) 

;j (ecase location 

jj ((:top-left :iop-right) 0) 

(( :botton-loft : bottom-right : bottom) *windoe-heigb.t*) ))) 
j! (s-value (agent -cursor 'self*) :viaible t) 
j! (update-honws *self*)) 
h (move- cursor •self* t y) 

'! *) 

I, 

(def request HOKE O 

Relinquish control of shared window and return hone. 
Always succeeds. 

If agent is on window when request issued, the agent is at hone 
and passive at end of request. 

[If agent is at hone, then request has no effect.] 



:i > 
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ii (unless (at-home? *s«U*) 

(with-process-lock (-lock*) 

(when (eq *»<reiv«* «self*) (deactivate-internal *snlf*))) 
1; (s-value (agent-cursor +s»ir*) rvisible nil) 

(oscillate •salt" nil) 
;■ (update -homos "self*) 

(update-»indovs)) 

i' tj 

i' (defrequast SIGNAL (flag) 

;; Torn signalling on or off. 
Succeeds iff agent is at bona. 
''\ After successful completion, agent is in urgent state if flag 

;; is nan-nil, othexHiae passive, 
t; (when (at-home? »self*) 
! (oscillate *self* flag) 

, t)) 

ij (def request INTERRUPT C) 

j, Deactivate currently active agent, if any. 

! ;; Succeeds iff there in an active agent [and returns that agent], 
jj ;; After successful completion, previously active agent is passive, 
ii (uith-process-lock (*lock*) 
j! (let ((agent »active*)) 

j ; (when agent 

i (deactivate-internal agent) 

■j (pushnev agent *p»ssive*) 

|! agent)))) 

j! (def request EXECUTE (action) 

]'. ;; Erecute given pointer action at current shared »indo» location, 
j! Succeeds iff agent is active and action succeeds, 

(unless (at -hone? *aelf») 
(vith-process-locx (*loci*) 
!! (when (eq "active* *self*) 

if (funcall action ♦■•If*))))) 

; i 

jj ;;; This io the currently the only action that can be called 

: ! ;;; from Execute above. 

<i 

ji (defun CLICK (agent) 

j! ;; Send button-click event to shared window at agent's current cursor location. 

; ; ALeays succaeds. 
|! (let ((x (agent-i agent)) 
Ij (y (agent -y agent))) 

i! (agent-send-event :buttcn-press :i x :y y 

j! :tine (♦ inter: : *last-tieie» inter :»double-click-time* 1>) 

■■ (agent-send-event : but ton -release :x i :y y)) 

{'. t) 

u 

j ■ i ; ; Queries [for information only, no state changes] 

j! (def request qOERY-AGEKTS () 

I ' 

-Ti- 
lt 

h 
I: 



j 
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;; Returns list of agents 
♦agents*) 

(dafrequest QUERY -ACTIVE () 

;; Returns active agent if any 
^active*) 

(defrequest QUERY-DISPLAY (agent) 

Returns r.on-nil if agent has a display (i.e., is human user) 
(agent-display agent)) 

(defrequest QUERY- LOCATION (agent) 

RaturnB (i . y) position on screen or nil if at home 
(unless (at-hona? agent) 
(cons (agent-x agent) (ag«nt-y agent)))) 

(defrequest QUERY-SIGNALLING (agenr) 

: ; Returns non-nil if I agent is in urgent state 
(agent-urgent agent)) 



TRANSLATOR 



;;; Whenever the pointer leaves the shared window (e.g. enters hone), 
;;; a HOME request is generated. 

(dafun leave-notify (window ftrest ignore) 
(declare (ignore ignore)) 

(when (• dlib:window-id window) (agent-window- id ♦self*)) 

(hone)) 
nil) 

;;; Whenever the pointer enters the shared window (e.g., leaves hone), a 
;;; HOVE request is generated to movm the user's cursor to that x/y 
;;; location. [The text area of the user's hone window is also 
;;; cleared.} 

(defun enter-notify (window i y krest ignore) 
(declare (ignore ignore)) 

((then (■ (xlib : window- id vindov) (agent-vindov-id »self*)) 

(set-cursor window (getf (xlib:wLndou-plist window) 'passive-x-cursor)) 
(let ( (*pixels-per-waxp* 10C0O)) 

(aove :y)) 
(clear-internal *nalf»)) 

nil) 

;;; Whenever the pointer noves within the shared window, a HOVE 
;;; request is generated with the new z/y location of the pointer. 

(defun cunor-aotion (window z y display) 
(declare (ignore display}) 

(when (■ (zlib: window -id window) (agent-window-id »self»)) 

- 24 . 
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Clot ((•pixels-per-warp* 10000)) 
(move j j))) 
;; to support dragging of ahov/hida-pointer 
! (when (getf (xlib:window-piist window) :poin-:er) 
i (intar: : mot ion-notify windou x y display))) 

I ;;; Whenever the user clicks on the shared window, an ACTIVATE request 
I ;;; i B goneratad. If the activate rtquMt succeeds, than 
j ;;; (EXECUTE ■click) request is generated and the application program wi 
1 ;;; see a button-click event at the location of the cursor. [If the 
j ;;; activate request fails. then the local display beeps. 1 

I (dafparanioter *meta-modif iar* 8) ;see garnet-kaytrans .lisp 

| (def parameter *meta-pr«f ix« (aref inter : : *pref ixes* *aeta-»odiIi<ir*) ) 

! (def parameter *neta-pref ix-longth* (length *mata-pref ix») ) 

j (defun panal-atop-action (intar obj) 

, (lot ((char (inter :e»ent-char inter:*current-svent*))) 

(if (and (symbol? char) 
! (string- "met a- prefix* 

(synbol-nane char) 
:<md2 •■eta-praf ix-length*)) 
| (call-prototype-nathod inter obj) : click froa execute request 

(elick-linal-funcxion)))) 

■ (defun click-final- function (toptional intar obj) 
final function lor application interactor 
(declare (ignore inter obj)) 
U (if (activate) 

(execute 'click) 
j! (intanboap))) 

(defun panel-select ion-function (gadget value) 
:> (declare (ignore gadget)) 
!• (vith-process-lock (*lock*) 
, synchronize panel on all displays 

jl (delist (agent *agent8*> 
j; (if-let (panel (agant-panel agent)) 

\, (a- value panel : value value)))) 

j| (update-aindovs)) 
< i 

Whenever the user clicks on the cursor pictura in his oan local 
home window, an SIGNAL request is generated with a true argument 
if the user is passive, or a false argument if the user is urgent. 

.' (defun nignal-seloction-f unction (gadget value) 

!! selection function for home-cursor 

i. (declare (ignore value)) 

\\ (let ((agent (g-valua gadget :parant -.agent))) 

; ! (when (eq agent •self*) 

J' (signal (not (agent -urgent agont )))))) 

;i ;;; Whenever the usar clicks on the remote hone windov of another 



i. - 
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;;; <tg«nt, if that agent is active, then an INTERRUPT rawest. 
; ; ; ie generated [othervise the local display beeps] . 

(defun interrupt-f inal -function (inter obj) 

final function for renote home interactor 
(d*clar* (ignore inter)) 
(let ((ag«nt (g-value obj : agent))} 
(yith-process-lock (»lock*) 
(cond ((eq agent *active«) 

(when *show* (show interrupt -text)) 
(interrupt) ) 
(t (int«r:boep)))))> 



Support Tor Controller Requests 



(defun at -home? (agent) 
j : (g-value (agent-homa-cursor agent) :visible)> 

t 

• (defun update-hoaes (agent) 

j! (update-if (agent-local agent)) 

!' (aapc t'opal :update (agent-roectos agent))) 

(defun update- windows () 
|i (dolist (agent -agents*) 
\- (update-if (agent -wind on agent));) 

i 1 

j| (def parameter *pixels-per-Barp* 10) 

ji 

:| (defun ■ove-cur&or (agent x y) 
i (let {(agent-i (agent-x agent)) 
jj (agent-y (agent-y agent))) 

(unless (and (= z agent-x) (= y agent-y)) 
!| (let* ({dx (- x agent-x)) 

j ! (dy (- y agent-y)) 

(steps (/ (oqrt (+ (• di dx) (* dy dy))) *pixela-per-warp*) ) 
]j (step-dx (funcall (if (ninusp dx) # 'floor »' ceiling) dx steps)) 

(step-dy (funcall (if (ninusp dy) f'floor t'ceiling) dy steps)) 
i; (abs-atep-dx (abs step-dx)) 

(abs-stap-dy (abs ot«p-dy))) 
>\ (dot iaea (n (round (1- steps})) 

<■ (warp- cursor agent 

|j (setq agent -x 

j| (if (< (abs (- x agent-r)) abs-atap-dx) 

• ; agwvt-x 

/> (♦ agent -i step-dx))) 

j, (eetq agent-y 

(if (< (abs (- y agent-y)) abu-atep-dy) 
1 1 agent-y 
;! (♦ agent-y step-dy)))) 

(unless (agent-display agent) (sleep 0.1))) 
j] (sarp-cursor agent x y))))) 

!i 

■■; • 2ti - 
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(dafun warp-cursor (agent x y) 

Clot ((cursor (agent-cursor agent))) 

(s-»alue cursor :19ft (- i (g-value cursor :hot-x)}) 
(s-value cursor :top (- y (g-value cursor :hot-y)))) 

(aetf (agent-x agent) x) 

(uetf (agent-y agent) y) 

(update-windows) ) 

(dof parameter •button-mation-mask* 

(xlib :make-evant-*»as)c :button-press :button-releaae :pointer-motlon)) 

(d«fun agftnt-send-event (»vent-key treat •vant-slots 

; , shared keys between button 
;, and notion events 
tUy (i 0) (y 0) (time 0) 
(root *x-root*) (root-x 0) (root-y 0) 
child (saae-screan-p t) (code 1) 
kallow-othtr-keys) 
(apply •'xlib:aend-«^«mt *x-window* e»ent-key ♦button-oot ion-mask* 
rwindow •x-vindow* :x x :y y :state *meta-moditier* 
:time tine :root root :root-i roor-x :root-y root-y 
: same-acreen-p same-scroen-p : child child 
:code coda event-slots) 
(display-output)) 



Hon* Windows 

* — * 

|; (d«f parameter *home -height* 120) 
>; (defparajn«ter *bome-vidth» 160) 

i' (defun renote-hone (agent window) 

(! ;; Create rejoote borne for agent on window 

!| ;; Return X-window 

i! (let* ((location (g-value (agent -bona -cursor agent) : location)) 

'. (hone (create-inatance nil inter: interact or-window 

■j (: agant agent) 

[ ! (:vindofl nindos) 

!j this fixes obscure tining problem with residtb 

i' (:doubl«-buf jered-p t) 

■[ (:titl« (agent-name agent)) 

(: height •ho»»-height*) 

>: (:width 'hoae-widthO 

Ij (:top (hose-top window location)) 

jj (:leix (home-left vindow location)))) 

t| (faco-hand (with-constanto-disablad 

!, (create* instance nil 

j' {g-value (agenTi-home-curaor agent) : parent) 

j: (:top (o-foreula (gv-cetiter-y-is-canter-of 

|! (gvl : parent)))) 

jj (: parts '((:faca :nodify (: visible t>) :hand))))) 
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(border (create- instance nil opal : rectangle 
(:top I) 
(:loft 1) 

(:widih (o-formula (1- (gvl :vindow :width)))) 
(:height (o-fornula (1- (gvl :uindo» :h«ight)))) 
(;line-style (create- instance nil opal :line-style 
(: line-thickness 2) 
( : foreground -col or 
(o-formula (g-value face-hand 

:hand :f illing-style 
: foreground-color) )))))) 

(top (create-instance nil top-aggregate) ) ) 
(opal : add- component top face-hand) 
(opalradd-component top border) 
(s- value home : aggregate top) 
(pu3b honw (agent-reswtes agent)) 

(with-opal -display (i)tb:a jndow-di splay (dravable window)) 

Copal: update home)) 
(create- instance nil inter :button-iatoractor 

(: Hindoo homo) 

C: agent agent) 

(: start -where '(:in .hone)} 

( iccmtinuous nil) 

( :f inal-f unction ' interrupt-f inal -function) ) 

(drawable home))) 

Cdefun local -hoae (agent location face passive urgent foreground -color text) 
(let* ((display (agent-display agent)) 

(gadget (create- instance nil local-gadget 

(:top (o-ioraula (gv-canter-y-is-center-of (gvl :parent)))) 
( :parta 
*((:home :modify 

(: parts (( : f ace-hajid 
:nodiiy 

{: agent .agent) 
(:f ace-prototype ,fac«) 
(: passive-image .passive) 
(: urgent- imago .urgent) 
( : foreground-color .foreground-color} 
(:f ace-on-right 
.(not (member location 

'(:top-left :bottom-laft))))) 
(:pan«l-arrov :modify (rvislble .text))))) 
:more :pencil)))) 
(hand (g-value gadget :hoao :f ace-hand : hand) ) ) 
(setf {agent-home-cursor agent) hand) 
(s- value hand : location location) 
(when display 

(let* ((window {agent-window agent)) 

(home (create- instance nil inter : interactor-windov 
(: agent agent) 
(:title (agent-name agent)) 
(.height +homa-height») 

>>i - 
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j 
i 

i 

{:uidth ♦home- width*) 

;; files obscure timing problem with rewidth 
( :double-buf f ered-p t) 
(:top {home-top window location)) 
<:left (home-left window location)))) 
(top (create-instance nil top-aggregate) ) ) 
(opal:add-camponant top gadget) 

(s-value (g-value gadget :hoae :face-hand :face) :visibl« nil) j 
(s- value home : aggregate top) j 
(aotf (agent- local agont) hone) | 
(with-op*l-display display t 

(opal: update hose)) 
(satf (agent- local -id agent) (ilib; window-id (drawabl* home))) 
(inter :warp-pointex hone 

thalf (g-value home :width>) 

(half (g-valuo hoaa : he ight) ) ) ) ) ) ) 

I 

(dofun homo-top (window location) j 
(scase location i 
((:top-right :top-left) 

(- (g-value window :top) (half "hoae-height*) ) ) , 
((: bottom : bottom- left : bottom- right) ! 
(- (opal :botton window) (half *home-height*) ) ) )) [ 

(d«fun home-left (window location) 
(ocaao location 

((:top-laft ;bottois-left) (- (g-value window :left) (half *ho»»-width«)) ) 
C(;top-right :bottom-rigbt) (- (opal: right window) (half ♦home-width*))) 
(tbottom (- (opal:center window) (half *hoaa-width*))))) 



Cursors 



(create-instance ' STIPPLED-FILLING-STYLE opal :f illing-styl* 
(:f ill-stylo :atipplod)) 

(defvar *f illing-style* 

(o-formula (create-instance nil etippled-f illing-atyle 

( foreground- color (diaplay-color-get (gvl :windou) 

<gvl :foreground-color))))>) 

(create-instanea ' CURSOR-BITMAP opal:bitnap 

( :f illing~»tyl« (o-formula (gvp :parant :f illing-style) >) ) 

{creata-instance "CURSOR my-gadgat-prototype 
(:top 0) 
(:laft 0) 

(;nidth (o-formula (gvl .'passive-cursor :width))) 

(: he ight (o-formula (gvl passive-cursor : height))) 

(: active nil) modified by requests 

(:viaibl« t) ;modlfied by raquaste 

(:hot-x 0) 



I 
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(:hot-y 0) 

( :passive- image user: :*rightup*) 

( :active-iaage us»r: :*rightup-rill») 

( :foreground-color dark-groon) 

( :filling-style (foroula *f i lling-style*) ) 

( : par is 

* (( :paasive-cursor .cursor-bir.map 

(: image r (o-fornula (gvp : parent : passive-inagu) )) 

(;top ,{o-formula (gvp :parent :top>)) 

(:left , (o-foniula (gvp :paxent :left))) 

{; visible ,(o-formula (and Cgvp ; parent : visible) 

(not (gvp :parent :active> )) ) ) ) 
( : active- cursor .cursor -bitmap 
(;inage . (o-foraula (gvp : parent : active- image) ) ) 
C:top . (o-fcraula (gvp : parent :top))) 
(:left .(o-foraula (gvp :par*nt :laft))) 
{: visible .(o-fornula (and (gvp : parent : visible) 

(gvp :parent ;activa)))))))) 

;; Note this priority nust be higher than *agent-pr iority* or oscillating 
; ; is atarvad out 

(d«f parameter *oscUlate-priority* 50) 

(defun oscillate (agent flag) 

(let ((oscillate (agent-urgent agent)) 

(process (agent-oseillate-process agent))) 
(cond ((and ilag (null oscillate)) 

(setf (agent-urgent agent) t) 
(cond ((and process (processp process)) 
(activate-process process)) 
(t (setf (agent-oscillate-process agent) 
(make- process 

:nane (string-append (agent-name agent) " Oscillate") 
:priority *0Bcillate-priority« 
: function 
•'(laxbda () 

(handler-bind 
((error # 'invoke-debugger) ) 
(loop 
(sleep 0.5) 

(s-value (ag»nt-hom«-cur»or agent) :urg«nt 

(not (g-value (agent-home-curaor agent) 
:urgent))) 
(update-hooes agent))})))))) 
((and (null flag) oscillate) 
(setf (agent-urgent agent) nil) 
(when (and process (processp process)) ' 

(doactivate-process process)) 
(s-value (sgent-hoae-cursor agent) :urgent nil) 
(update-hones agent))))) 



- Mi 
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j) Text Communication 



l! 



(create-instanea ' * GEM T- TEXT multi -string-text 
'} (:font (opal :g*t-atandard*font :sana-serlf :bold :mediu«s))) 

i | 

j! (dafun agent-text (krast strings) 
■) (create-instanca nil agent-text 
!! (:atrings atrings))) 

(craate-instanca * IMTERR17PT-TEXT agent-tilt 
(: strings ' ("Interrupt . "))) 

!' (def request SHOW (gob) 

ij (when *shoB* 

jl (unlaao (is-a-p gob opal : graphical -object) 

j! (setq gob (apply f 'agont-text (if (stringp gob) (list gob) gob)))) 

!■ (let* ((on-right (g-value (agent -home- cure or »salf*) : f ace-on-right) ) 

j! (top (hall (- * hone- height* (g-value gob iheight)))) 

I! (left (if on-right (♦ ♦hom«-vidth» B} 8» 

j! (justification (if on-right ileft :right>) 

I; (remotes (agent-remotes •self*}}} 

j! (unlasa (and remotes (is-a-p (g-value (first renrataa) :ahov) gob)) 

j! (clear-interna.1 *self* on-right) 

|j (dollst (window remotes) 

ij (vith-opal-display (window-display window) 

!j (opal;add-component (g-valua window : aggregate) 

1 1 (a-value window :shou 

i: (creata-instance nil gob 

!i ( :top top) 

Ij (:left loft) 

,j (: justification justification)))) 

1 1 (rewidth window (♦ (g- value gob : width) *home-width* 16) 

jj on-right))))))) 

jj (def request CLEAR () 

,! (clear-internal *s«li»)) 

;j 

i| (defun clear- internal (agent koptional 

! (on-right (g-valua (agent -homo- cursor agent) :f ace-on-right) )) 

ij (dolist (window (agent -remotes agent)) 
■ j (1st ((gob (g-value window :show))) 

jj (when gob 

|i (opal:r«nove-companent (g-value window : aggregate) gob) 

jj (opal :destroy gob) 

(s-value window :show nil) 
(rewidth window *hoiaa-width* on-right))))) 

(defvar •user-send* nil "Variable to hold item sent by user") 

(dofun user-sand (gadget item) 
(declare (ignore gadget)) 
(setq ♦ucer-oend* item) 



01/09/2003, EAST Version: 1.03.0002 



5.796.396 



41 



42 



{ahoM {string-append itnn "."))) 

i! 

i; 

Demonstration 



1? 



:| 



jj Ureatu-iiistajice 'DEHD-PANEL gg:mot,i/-lexl-bult.on-p<uiel 

.1 (:top (o-fornula <- (gv-etnter-y-is-center-of (gvl :parent)) SO))) 

;j (.left Co-formula (gv-center-x-is-center-of (g»l :parent)))) 

; ! (;f<jnt Co- formula (gv nygadget-prototypo :bold))) 

;■ ;; get version of motif-gray on this local display 

| ! (:foreground-color Co-formula 

.j (display-color-get (gvl :vindow) opal .rnoti f -gray ) )) 

(litems '< "Button 1'" "Button2" "Button3" "Buttonl" "Buttons" 
j: "Button6" "Button7" "Button8" "Buttons" >) 

( :f inal-f eedback-p t) 
>: C:toggl«-p t) 

'J (:text-offset 10) 

;l (:v-sp»cing 10) 

(:h-spacing 10) 
!■ (:dIrection horizontal) 

l| C : rank -marg in 3) 

( :selection-Iunction 'panel-selection-f unction) 
( : infractors 
jj '({: press : modify 

'i (: continuous nil) 

jj <: aalting-priority . inter:high-priority-level) 

j: (: start-event :any-leftdotm) 

;f (: stop-action 'panel-stop-action)))) ) 

,[ (daivar *glurk-likes* nil "Button that Glurk likes") 

jl (defvar ♦glurx-hates* nil "Button that Clurk hates") 

il 
i: 

jj ;;; Clurk thinks setting Button6 is a bad idea; whenever he sees 

i! ;;; Buttons set, be tries to clear it. Clurk likes Buttons; whenever 

jj ;;; another button is set, ho suggest setting Button9 by putting a 

|j ;;; passive finger on it. Glurk also likes to play with the 

j ;;; buttons. Whenever there are no buttons set, he starts to press 

jj ;;; each button in turn, but Hill stop if he sees signalling from 

:1 ;;; either of the two humans. 

i! 

j| (create- instance ' ALL-BUTTONS-TEXT agent -text 

jj (: strings '("I'd like to try all the buttons." 

!| "CK?">)> 

jj (create- instance 'HATE-TEXT agent-text 

;! (: strings '("I bate Button7." 

!j "I'm going to turn it off!"))) 

I ; 

;, (create-iastan.ee 'LIKE-TEXT agent-teit 

j' <: strings '("Button9 vould be better."))) 

l| (defun glurk-loop () 

il 
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(when *panel» 

(cond ((equal *uaar-send» "Try all the buttons") 
(try-all -buttons)) 

((and (query-signalling *self*) (if 'shoe* 

(equal *UBer-aend* "Ok") 
(query-active))) 

(signalling nil) 

(shot! all-buttons-text) 

(when (if »6how (ok-or-no) t) 

( t ry - all - buttons ) 

(ho»e))) 
{(g-valua ♦panel* : value) 
(signalling nil) 

(cond ((g-value *gluTlc-hatas» selected) 
1 (show hate-text) 

! (when (activate) (clear -button ♦glurk-hates*) 

j (clear) 
I (home))) 

| ((g-value *glurk-lihea* selected) 

! (clear) 

j (home)) 

j (t (show like-text) 

. (move -but ton *glurk-HX*B*)))) 

\ (t (clear) 

j (hone) 

I (signalling t)))) 

: ; (sleep I)) 

\ (dofvar •smurk-likes* nil "Button that Snurk likes") 
i (def vax *saurk-hates* nil "Button that Saurk hates") 

j 1 ;;; Smurk hateii like Button7; whenever he sees it set. he trios to clear it. 

ii 

ji (dafun snurk-ioop () 

i. (when (and *panel* (g-value *Bmurk-hates* ; selected) (not ( query-active) )) 
| ' (move-button *amurk-hatoo») 
,i (activate) 

1 (execute 'click) 

I (hone)) 
(sleep t)) 

! (d«fun ok-or-no () 

'[ (»«tq #u»er-*«ad* nil) 

!| (progl 

if (loop 

i: (cond ((e^ual «user-*end* "Ok") (return t)) 

ji ((equal *us»r-send* "Ho") (return nil)) 

! {♦us«r-aend* (aatq »ua«r-sand* nil))) 

j (sleep 1» 

! (clear) 

I (sotq •user-send* nil))) 

I! 

jj (dafun try-all-buttons () 

Ii 

i 1 ■ :u 

!j 



Si 

!i 

ii 
I 
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(setq ■user-send* nil) 
{clear) 

(let ((1 (g-value *panel* :button-list : components) ) ) 
(nove-button (first 1)> 
(activate) 
(catch "abort 

Cdolist (button 1) 
(abort?) 

(move-button button) 
(abort?) 

(execute 'click) 
(process -all OB- schedule) 
(sleep O.S) 

(unless (eq button #glurk-likes*) 
(abort?) 

(move-button button) 
(abort?) 

(execute 'click)) 
(process -allow- schedule) 
(sleep 0.5))) 
(hone))) 

(defun abort? () 

[when (or (not (eq (query- active) «solf#)) 

(if-let (agent (find •'query-signalling (query-agents))) 
(not (eq agent *self*)))) 



jj (throw 'abort t))) 

:{ (defun set-button (button) 

H returns non-nil iff succeeds 

:j (unless (g-value button : selected) 

1 1 (oove-button button) 

|| (execute 'click))) 

r (defun clear-button (button) 
i! returns non-nil iff succeeds 

|j Cohen (g-value button : selected) 
i; (aove-button button) 
(execute 'click))) 

(defun move -butt on (button) 
; ; always succeeds 

(let {(box (g-value button ; but ton))) 
(move (center-x box) (contor-y bcx)))) 

(defun find-button (string) 

;; Returns instance of gg:notif -text-button 
(find string 

(g-value *panel» :button-list :coatponents) 
j! :key •' (lambda (button) (g-value button :iten-obj)) 

i j :t«at '« 'equal)) 



■ :t4 - 
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In summary, the above describes a system in which 
multiple human users and/or software agents interact with a 
single, shared application window, such that each user or 
software agent can easily tell who is providing input at any 
given moment, and such that input permission is conve- 
niently and automatically granted and relinquished. In the 
embodiments described above, the operation of the auto- 
matic input-permission system is divided between a con- 
troller module, which contains the essential rules for grant- 
ing and relinquishing input permission, and a translator 
module, which maps users' graphical motions into controller 
requests. This decomposition makes it possible for software 
agents to participate in window sharing on the same basis as 
human users. Furthermore, in embodiments with Home 
Windows, the correspondence between users or software 
agents and their cursor icons is made more evident, as well 
as facilities provided for text communication between users 
and/or software agents. 

Having above indicated a preferred embodiment of the 
present invention, it will occur to those skilled in the art that 
modifications and alternatives can be practiced within the 
spirit of the invention. It is accordingly intended to define 
the scope of the invention only as indicated in the following 
claims. 

What is claimed is: 

1. A system for the control of a single program running at 
one location, said program shared by users of a network of 
associated terminals, each terminal having a display screen, 
a dedicated window at said display screen and window input 
event generating means, the results of running said shared 
program being displayed at each terminal through the use of 
the window thereat, comprising: 
means coupled to said network at said one location for 
both storing and executing said single program and for 
storing data associated therewith such that said single 
program and associated data storage are co-located and 
such that said storing and executing means provides a 
single running program shared by said users; 
means including a software agent and coupled to said 
network for automatically permitting a window input 
event from one user to control said shared program 
such that said program is under the control of said one 
user; and, means for indicating to all of said users 
which of said users has been granted permission to 
control said shared program. 
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2. The system of claim 1. wherein said indicating means 
include means for indicating to all users when any of said 
users is seeking input permission. 

3. The system of claim 2. wherein said automatic input- 
5 permission establishing means includes means for ascertain- 
ing when any of said users is seeking input permission. 

4. The system of claim 1. wherein said indicating means 
includes means for generating an on-screen cursor icon 
unique to each user at each of said display screens such that 

10 each of said icons is replicated on all of said display screens. 

5. The system of claim 4. wherein said indicating means 
includes means for altering the appearance of said on-screen 
cursor icon, depending on whether said user has input 

l5 permission. 

6. The system of claim 4. wherein said indicating means 
includes means for altering the appearance of said on-screen 
cursor icon, depending on whether said user desires input 
permission. 

^ 7. The system of claim 1, wherein said indicating means 
includes indicia at each screen for indicating on-screen 
which of said users has been granted input permission. 

8. The system of claim 1, wherein said display screen 
includes on-screen control icons, and wherein said means 

^ for automatically establishing input permission includes 
means available to each of said users and including a mouse 
for moving a cursor associated with a user over said display 
screen and for actuating an on-screen control icon by 
overlaying said control icon with said cursor and clicking 

w said mouse. 

9. The system of claim 4. and further including means for 
generating home windows on each of said display screens, 
each associated with a different user, at which the on-screen 
cursor icon associated with said user initially resides. 

J5 10. The system of claim 9, wherein said means for 
automatically establishing input permission includes means 
for moving an icon from said home window to said window. 

11. The system of claim 1. and further including a 
software agent and means for coupling said software agent 

w to said means for automatically establishing input permis- 
sion. 

12. The system of claim 1. wherein said indicating means 
includes means for generating an on-screen cursor icon 
unique to said software agent 

* * * * * 
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